Procedia Computer Science Conference on Systems Engineering Research (cser'13) Systems Engineering Research Methods

One of the challenges of systems engineering research is that the expertise and the application happen "in the field". The field can be an industrial company or a government agency. Researchers need methods to research in the field; methods to try-out ideas, collect data, analyze, and evaluate. In this paper, we present a simple research model to help researchers to shape their research. We provide and discuss a number of elementary research methods. The model and the elementary research methods are based on our systems engineering research experiences in the past five years. We conclude the paper with some validation challenges and pitfalls. © 2013 The Authors. Published by Elsevier B.V. Selection and/or peer-review under responsibility of Georgia Institute of Technology. Systems Engineering; research methods 1. Systems Engineering Research Systems Engineering is a young discipline that is mostly practiced “in the field”, e.g. in industrial companies. Many systems engineers do their work based on experience [1]. An experienced group of INCOSE editors filled the Systems Engineering Body of Knowledge (SEBoK) [2] with a rich collection of best practices. Hot questions in the systems engineering community is what the return on investment is of applying systems engineering [3], or what the value is of applying systems engineering [4]. More general the academic question is “how do we know the effectiveness of these best practices?” In this paper, we will discuss research methods to study effectiveness of systems engineering methods and techniques in practice. We focus on field research, under the assumption that researchers can only observe effectiveness in practice in the field. Dominant research approaches to study in the field are action research [5] and industry-as-laboratory [6, 7]. The researcher combines active participation in the systems engineering activities with the researcher role. The researcher is wearing two hats in these research approaches: as systems engineer and as researcher. The attitude of the systems engineer is result oriented and cooperative. The attitude of the researcher is questioning and challenging. Where the systems engineer will promote the use of a proposed method or technique, the researcher needs to question its validity. * Corresponding author. Tel.: +4732869594; fax: +4732869551. E-mail address: [email protected]. Author name / Procedia Computer Science 00 (20123) 000–000 Researchers in systems engineering are typically educated in the technical domain. However, effectiveness of systems engineering depends largely on human aspects, such as competence and behavior of individual stakeholders, social interaction between stakeholders, political circumstances, organization and governance, and many more. Research in systems engineering has to build on available scientific methods, both technical, as well as from the social sciences. Bhattacherjee wrote good introduction in social science research [8]. Figure 1 shows the logical order of steps to define a research project in systems engineering. The starting point is a need for improvement in the field, triggered by an industrial problem. Researchers reformulate the problem into an industrial goal; “where do we want to go” instead of “where are we”. The problem triggers multiple research questions, e.g. how much can method x mitigate the problem? The next step is to sharpen the research by looking for quantifiable propositions, e.g. full requirements traceability will reduce the change request rate after the project definition with 80%. Researchers can use the research questions and the quantified propositions to formulate a hypothesis as basis for evaluation. Often a number of explicit criteria help in such evaluation. Preferably, the research leaves room to study multiple options. The researcher will need some baseline to evaluate. The baseline can be past performance, benchmarking with other projects or organizations, or comparison of different solutions. research industrial problem